Image matching server network implementing a score based on a function of latency between a server and an image store

ABSTRACT

Technology is disclosed for feeding source images from image stores of client systems into an image processing system using image feeding servers (“the technology”). The technology includes multiple image feeding servers that can feed a source image to the image processing system, using which the image processing system can generate processed images to be served to end users. The image feeding servers can be installed at various locations, e.g., geographically spread, and more proximate to where the source images are stored. An image feeding server obtains the source image from the image store in original size, downsizes the source image, and transmits the downsized source image to the image processing system. The technology selects an image feeding server based on a score of the image feeding server. The score can be determined as a function of one or more image feeding server parameters, e.g., latency, workload, or computing resource.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.15/861,053, titled “IMAGE MATCHING SERVER NETWORK IMPLEMENTING A SCOREBASED ON A FUNCTION OF LATENCY BETWEEN A SERVER AND AN IMAGE STORE,”filed Jan. 3, 2018, which is a continuation of U.S. patent applicationSer. No. 14/332,063, titled “IMAGE FEEDING SERVER NETWORK,” filed Jul.15, 2014, now U.S. Pat. No. 9,894,023, issued Feb. 13, 2018, which isincorporated herein by reference in its entirety.

TECHNICAL FIELD

Several of the disclosed embodiments relate to image processing, andmore particularly, to feeding an image from a source to an imageprocessing system for just-in-time image processing of the image.

BACKGROUND

Online publishers publish various content including text, image, videoand audio. An online publisher may want to publish a particular image invarious color schemes, patterns, layouts, etc. For example, an onlineart merchant may want to publish a sculpture image captured using acamera online. However, the merchant may want to enhance the image in aparticular way before the image is published online to demand a buyer'sattention. The merchant may want to generate multiple images showingspecific portions of the sculpture.

Current publishing techniques typically require the merchant to generatea version of the particular image for each of the different ways themerchant wishes to publish the image. For example, if the merchantwishes to highlight only the portion of the image having the sculpture,the surrounding portions of the sculpture in the image may have to beblurred. In another example, if the merchant wishes to show how thesculpture looks in a particular decor, the surrounding portions of thesculpture in the image may also have to be shown. This requires themerchant to have different version of the images which serves differentpurposes. This can be a time consuming process since all versions haveto be generated before they are published.

Certain service providers provide image processing services to theonline publishers. The image processing services typically obtain asource image from the online publisher and perform processing the sourceimage to generate various versions of the image. The source images aretypically large files, e.g., ranging from tens of megabytes to hundredsof megabytes. To transport such large image files from the onlinepublisher systems to systems of the image processing services consumesignificant computing resources, e.g., network bandwidth, and can resultin increased network consumption costs. The problem increases by manyfolds if the images published by the online publishers are changingoften as the image processing system has to obtain the source imagesoften. Further, in many cases such large source image files may not benecessary to generate the required processed images. For example, if theprocessed images are typically presented on smaller screen mobiledevices, then obtaining the source image, which is a high resolutionimage and is of hundreds of megabytes may not be necessary and is not anefficient use of the computing resources of the image processing system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an environment where an image processing system canbe implemented.

FIG. 2 illustrates a block diagram of a system for obtaining downsizedsource images using image feeding servers, consistent with variousembodiments of the disclosed technology.

FIG. 3 is a block diagram of an image processing system of FIG. 1,consistent with various embodiments.

FIG. 4 is a flow diagram of a process for processing an image,consistent with various embodiments.

FIG. 5 is a flow diagram of a process for selecting an image feedingserver to obtain the downsized source image, consistent with variousembodiments.

FIG. 6 is a flow diagram of a process for feeding a downsized sourceimage to an image processing system, consistent with variousembodiments.

FIG. 7 is a flow diagram of a process for determining a score of animage feeding server, consistent with various embodiments.

FIG. 8 illustrates a block diagram of an image processing systemimplemented as a multi-tier architecture, consistent with variousembodiments.

FIG. 9 is a block diagram of a computer system as may be used toimplement features of some embodiments of the disclosed technology.

DETAILED DESCRIPTION

Technology is disclosed for feeding an image to an image processingsystem for just-in-time processing of the image (“the technology”).Various embodiments of the technology include an image processing systemfor providing image processing services for clients who publish content,including a number of images, to users. In some embodiments, the imageprocessing system processes the images upon receiving a request forretrieving the images associated with the content. That is, the imageprocessing system processes the images just-in-time. For example, whenthe user accesses a website or an application (“app)” of a client systemfor viewing the online content, the client system requests the imageprocessing system to obtain the images associated with the content. Theimage processing system processes the images and returns the processedimages to the client system, which further serves them to the requestinguser device.

In some embodiments, processing an image includes executing a set ofinstructions (also referred to as a “recipe”) on one or more originalsource images to render a processed image on the user device. Typically,the image processing system obtains the source image from the clientsystem and uses it to generate one or more processed images.

Various embodiments of the technology include multiple image feedingservers that can feed a source image to the image processing system,using which the image processing system can generate the processedimages. The image feeding servers can be installed at various locations,e.g., geographically spread, and more proximate to the client systemswhere the source images are stored. An image feeding server obtains thesource image from the client system in original size, downsizes thesource image, e.g., decreases the file size of the source image bydecreasing a resolution of the source image, and transmits the downsizedsource image to the image processing system. The image processing systemuses the downsized source image to generate one or more processedimages, which can be served to end users accessing content published bythe client system. In some embodiments, downsizing an image meansreducing a file size of the image. The file size can be reduced in anumber of ways, e.g., reducing the resolution, that is, the number ofpixels of the image, changing a file format of the image file,compressing the image using various compression techniques, or reducingthe number of colors in the image.

In some embodiments, in fetching a source image from a client system,the image processing system can choose an image feeding server that ismost efficient, e.g., consumes the least amount of computing resources,to fetch the source image, downsize the source image to the specifiedfile size and transmit the downsized image to the image processingsystem. For example, the image processing system can choose an imagefeeding server which is closest to the client system, that is, an imagefeeding server that has a least latency to the client system among allthe image feeding servers. In another example, the image processingsystem can choose an image feeding server that has the most workcapacity or the least work load among all the image feeding servers. Inyet another example, the image processing system can choose an imagefeeding server based on the latency of the image feeding server and thework load of the image feeding server.

After a particular image feeding server is chosen, the image feedingserver obtains the source image from the client system, downsizes theimage and sends the downsized image to the image processing system overa high speed network. The latency of the high speed network issignificantly lower than that of the latency of a network between theimage processing system and an image store of the client system (wherethe client system stores the source images) in the absence of the imagefeeding servers. The latency between the particular image feeding serverand the image store of the client system is also significantly lessercompared to the latency of a network between the image processing systemand the client system in the absence of the image feeding servers.

Further, the combined latency between the image store of the clientsystem and the particular image feeding server and between theparticular image feeding server and the image processing system is stilllesser than the latency of the network between the image processingsystem and the image store of the client system in the absence of theimage feeding servers. Accordingly, the computing resources, e.g.,network bandwidth, time, consumed in fetching the downsized source imageto the image processing system is significantly less compared toobtaining the original source image in the absence of the image feedingservers.

The image processing system can further process the downsized image togenerate one or more processed images as per client system's requirementand/or end user device's attributes. In some embodiments, the downsizedimage is large enough to generate any of the one or more processedimages that may be presented to the end user device and small enough tobe transmitted from the image store of the client system to the imageprocessing system without significant consumption of computingresources. The image processing system can specify a target file size towhich the source image has to be downsized to in the request sent to theimage feeding server. In some embodiments, the target file size is alowest possible file size of the source image using which a downsizedimage, which can be used to generate the above one or more processedimages, can be generated.

In some embodiments, the image processing system processes the downsizedimage based on a recipe to render a processed image on the end userdevice. The recipe enables the processed images to be rendered on theend user device as per the preferences of a client system using theimage processing system. For example, the recipe can specify that theprocessed images be rendered in a particular size, color scheme, layout,resolution, etc. The recipe can also specify that a portion of theprocessed image be highlighted, cropped, a particular color be enhanced,etc. In some embodiments, the recipe may be provided, at least in part,by the client system. The recipe may also be generated, at least inpart, by the image processing system. The image processing systemenables the client system to obtain various processed images for a givensingle original source image. In some embodiments, more than one sourceimage may be used to render the processed image.

In some embodiments, the images are also processed based on a context ofthe request. The context of the request can include data regardingattributes of the end user device, e.g., a resolution of display, a sizeof the display, a type of the end user device—a smartphone, tablet,laptop, desktop, etc., an operating system on the end user device,ambient lighting, a network bandwidth available to the end user deviceetc. The image processing system renders the processed images based onthe attributes of the device. For example, the images can be rendered ina particular resolution based on the resolution of the display of thedisplay of the end user device. In another example, the brightness ofthe processed image may be adjusted based on the ambient lighting on theend user device. In some embodiments, the context of the request forobtaining the image is included in the request and/or provided by theclient system.

In some embodiments, the image processing system can be implemented as amulti-tier architecture having various tiers of servers performingspecific functions of the image processing system. Each of the tiers canhave one or more servers. The image processing system can include acontent delivery network having one or more tiers to receive a requestfor obtaining an image to be displayed at the end user device from aclient system. The servers in the content delivery network can be spreadacross various geographical locations. The content delivery networkreceives the request from client systems in various geographicallocations passes the request to a load balancing server tier if theprocessed images are not available at the content delivery network.

The load balancing server tier manages image processing requests fromthe client system by inserting them into a queue. The load balancingserver tier receives the request from the content delivery network andinserts the request into the queue. An image processing server tierobtains an image processing request from the queue and generates aprocessed image. The processing includes retrieving one or more originalsource images and processing them based on the recipe and the context ofthe request to generate the processed image. The processed image isreturned to the client system through the content delivery network andthe client system further serves the processed image to the end userdevice.

In some embodiments, the image processing system also includes a sourceimage server tier that is configured to obtain the original sourceimages, e.g., downsized source images, from a source location using theimage feeding servers, and store them at a source image cache associatedwith the source image server tier. In some embodiments, the imageprocessing system generates a log file containing data related to theprocessing of the one or more requested images. The image processingsystem also includes a prediction server that can predict the differentprocessed images that may be requested by the end user devices in thefuture. In some embodiments, the prediction server can use variousprediction techniques, including statistical analysis, to determine aset of processed images that may be requested in the future based, atleast in part, on the data in the log file.

In some embodiments, the image processing system provides an imageediting tool (hereinafter referred to as a “visor”) that generates arecipe for on-demand processing of images. The visor includes agraphical user interface (GUI) that provides various editing controlsfor performing various editing operations on an image. The client systemcan use the visor to edit the image. The visor generates a set of imageprocessing instructions corresponding to the editing operationsperformed by the client system and transmits the set of image processinginstructions, which form a recipe for generating a processed image, tothe image processing system. The image processing system processes theimage based on the recipe and transmits the processed image to theclient system which further serves the processed image to the end usersaccessing the image.

In some embodiments, the image can be part of the content published bythe client system. For example, the image can be part of contentpublished via a webpage. In some embodiments, the image processingsystem generates the visor on the same webpage as the content having theimage. By generating the visor on the same webpage as the content, theclient system can edit the image on the webpage and view the resultingimage as it will be seen by an end user accessing the content. This canhelp the client system in obtaining a perspective on whether the editedimage may be pleasing to the end users' eyes and in determining whetheror not to publish the edited image.

Environment

FIG. 1 illustrates an environment where an image processing system canbe implemented. The environment 100 includes a client system 115 thatpublishes content, including images, which can be accessed by end usersusing end user devices such as end user devices 105 a-c via acommunication network 110. The end user devices can be a smartphone, alaptop, a desktop, a tablet PC, or any other suitable device for viewingcontent published by client system 115. The environment 100 alsoincludes an image processing system 125 that receives a request from theclient system 115 (e.g., on behalf of an end user device such as enduser device 105 a) for obtaining a processed image (of a source image)which is served to the end user device 105 a with the content. The imageprocessing system 125 obtains the source image and processes the sourceimage to generate a processed image (also referred to as “renderedimage”). The processed image is then returned to the client system 115which further serves the processed image to the end user device 105 a.

The source image most often provides the base image data used inproducing a processed image. In some embodiments, one or more sourceimages may be used in generating the processed image. In someembodiments, the image processing system 125 obtains the source imagefrom a location such as an image store 120 specified by the clientsystem 115 or from a cache associated with the image processing system125 (not illustrated). The image store 120 can be any type of storagesystem including a database, file storage and/or any other convenient orknown storage techniques. Further, the image store 120 can be part ofthe client system 115 or provided by any third party storage services,such as cloud storage from Box, Dropbox, Amazon, Google, etc., where theclient system 115 stores the source images.

In some embodiments, the image processing system 125 obtains a downsizedversion of the source image and processes the downsized source image togenerate the processed image. The image processing system 125 obtainsthe downsized version of the source image using one of the image feedingservers 150. One of the image feeding servers 150 obtains the sourceimage in original file size from the image store 120, downsizes thesource image to a target file size specified by the image processingsystem 125 and transmits the downsized source image to the imageprocessing system 125. Additional details with respect to obtaining adownsized source image using the image feeding servers are described atleast with reference to FIGS. 2-7.

The image processing system 125 processes the downsized source imagebased on a recipe for the particular image and/or a context of therequest. The recipe includes a set of instructions to be executed on thesource image to generate the processed image. The recipe can includeinstructions regarding at least one of (a) a target size of the image tobe rendered, (b) a target resolution of the image to be rendered, (c) atarget color pattern of the image to be rendered, (d) a target layout ofthe image with respect to other content with which the image is served,(e) a target brightness value, (f) a target contrast value, (g) a targetsaturation value, (h) a target portion of the image to be highlighted,or (i) a target area of the image to be cropped etc.

In some embodiments, the recipe can be provided by the client system115. For example, the client system 115 can provide a recipe whichinstructs the image processing system 125 to crop an image to includeonly a portion of the image having faces of certain subjects. In anotherexample, the recipe can instruct the image processing system 125 to cropa specific portion of the image if the image is rendered on asmartphone, but include a larger portion of the image when it isrendered on a laptop. In another example, the recipe can instruct theimage processing system 125 to change a color pattern of the image inorder to match with a webpage with which the image is displayed or tohighlight/enhance a particular color of the image to grab attention fromthe end user.

In some embodiments, the recipe can also be generated by the imageprocessing system 125. For example, the image processing system 125 candetermine based on historical data, which of the colors when enhanced inthe image received more attention from the users (determined based on,e.g., a number of clicks) and enhance the color of the image and/orother images in the webpage accordingly. The recipe can be stored in astorage system such as a recipe store 130. In some embodiments, therecipe store 130 can be implemented as various storage systems includinga database, a file system and/or any other convenient or known storagetechniques.

In some embodiments, the recipe can also take into consideration acontext of the request. The context of the request can include anattribute of the end user device 105 a, such as (a) a type of the enduser device 105 a, (b) a size of a display of the end user device 105 a,(c) a resolution of the display of the end user device 105 a, (d) anoperating system of the end user device 105 a, (e) ambient lighting ofthe end user device 105 a etc. For example, if the end user device 105 ahas a display with resolution higher than a predefined resolution, thenthe recipe can instruct the image to be rendered in a resolution higherthan a predefined resolution. In some embodiments, the image processingsystem 125 can have a predefined range of resolutions at which theprocessed image can be rendered. In another example, if the end userdevice 105 a has a display with resolution lower than a predefinedresolution, then the recipe can instruct the image to be rendered in lowresolution. In another example, the brightness of the image can bechanged based on the ambient lighting condition on the end user device105 a.

Further, in some embodiments, the context of the request can alsoinclude network bandwidth availability between the end user device 105 aand the client system 115. For example, if the network bandwidth isbelow a predefined threshold, the recipe can instruct the imageprocessing system 125 to render a low resolution image. The context ofthe request can be included in the request and/or provided by the clientsystem 115 independent of the request.

In some embodiments, various recipes can be used to generate variousprocessed images for a particular image. The client system 115 mayprovide access to at least one source image to the image processingsystem 125 and request the image processing system 125 to return one ormore processed images for the source image. For example, for a givenimage by the client system 115, various processed images such as a highresolution image, a low resolution image, an image with a particularcolor enhanced, an image with a particular color scheme, brightness,saturation or hue etc. can be generated using the corresponding recipe.In some embodiments, a given recipe can be reused for processing variousimages. For example, a default recipe can process one or more images.The default recipe can be for enhancing the colors, brightness,saturation, etc. of the one or more images.

The image processing system 125 enables the client system 115 to obtainvarious processed images for a given single image. This minimizes theresources required for maintaining various versions of the images by theclient system 115. However, in some embodiments, more than one sourceimage may be used to render a processed image, for example, a derivativeimage. A derivative image is a kind of rendered image that depends onone or more original source images. For example, an image of a logo maybe superimposed onto another image to obtain the derivative image. Inembodiments where the image processing system 125 renders a derivativeimage, the image processing system 125 may retrieve the necessaryoriginal source images from the image store 120.

The content published by the client system 115 can include text, video,audio, or any other multi-media content. The content can be published inmany ways, including websites, applications (also known as “apps”). Thecontent can be accessed by the end users in a number of ways, e.g., viaweb browsers, or apps which when executed on an end user device displaythe content.

In some embodiments, the image processing system 125 can be implementedas a multi-tier architecture having various tiers of servers performingspecific functions of the image processing system 125. One such examplemulti-tier implementation of the image processing system 125 isillustrated in FIG. 8.

FIG. 2 illustrates a block diagram of a system 200 for obtainingdownsized source images using image feeding servers, consistent withvarious embodiments. The system 200 may be implemented in theenvironment 100 of FIG. 1. The system 200 includes multiple imagefeeding servers, e.g., image feeding servers 150 a-f (referred to asimage feeding servers 150), that can feed downsized source images to theimage processing system 125. The image processing system 125 generates ajob request for obtaining a source image, e.g., source image 215,advertises it to the image feeding servers 150, and assigns it to one ofthe image feeding servers 150, e.g., image feeding server 150 a. Theimage feeding server obtains the source image from the particular imagestore, e.g., image store 120, in original size, downsizes the sourceimage, and transmits the downsized source image, e.g., downsized sourceimage 220, to the image processing system. By way of example, a sourceimage can be 200 MB and the corresponding downsized image can be 100 KB.In some embodiments, the image processing system 125 generates the jobrequest for obtaining the source image 215, when the source image 215(or the corresponding downsized image 220) for generating a processedimage in response to a request from the client system is not stored atthe image processing system 125.

In some embodiments, the image feeding servers 150 are geographicallyspread and can be located near image storage systems, e.g., image stores120, 210, and 220, of one or more clients. To assign the job request toan image feeding server, the image processing system 125 selects aparticular image feeding server based on one or more factors, e.g.,proximity (or connection time or latency) to a particular image storefrom the which the source image has to be fetched. For example, to fetchthe source image 215 from the image store 120, the image processingsystem 125 can select the image feeding server 150 a that is inproximity/has lowest connection time to the image store 120. Afterobtaining the source image from the image store 120, the image feedingserver 150 a downsizes the source image and transmits the downsizedsource image 220 to the image processing system 125. The followingparagraphs describe additional details with respect to the process ofselecting a particular image feeding server to obtain a source imagefrom a particular image store.

The image processing system 125 selects an image feeding server based ona server score of the image feeding server. The server score isindicative of a match of the image feeding server to perform the jobadvertised. The server score can be determined as a function of one ormore image feeding server parameters, e.g., latency, workload, computingresource, of an image feeding server. In some embodiments, the serverscore is determined as a function of latency between a particular imagefeeding server and a particular image store from which the source imageis to be obtained. Latency is a time interval between the stimulationand response, or a time taken for a data packet to travel from source todestination and back. In some embodiments, the lower the latency, thehigher the server score.

For example, for obtaining a source image from image store 120, theimage feeding server 150 a can be the most proximate image feedingserver to the image store 120 and therefore, can have a least latency tothe image store 120 among the image feeding servers 150 andconsequently, the highest score. Accordingly, the image processingsystem 125 can select the image feeding server 150 a to obtain thesource image from the image store 120. In another example, with respectto image store 210, the image feeding server 150 e may have a leastlatency to the image store 210 among the image feeding servers 150.Accordingly, the image processing system 125 can select the imagefeeding server 150 e to obtain the source image from the image store210. In some embodiments, the server score of an image feeding servercan be different for different image stores as the image feeding servercan have different latencies with different image stores, e.g., as theimage stores may be in different regions.

In some embodiments, the server score is determined as a function of theworkload (or available capacity) of the image feeding server. In someembodiments, the lesser the work load (that is, higher the availablecapacity) of the image feeding server, the higher the server score. Thework load of the image feeding server can be determined as a function ofnumber of tasks, e.g., downsizing tasks, the image feeding server isperforming and a file size of the source image being downsized in eachof the downsizing tasks. For example, if two image feeding servers havesubstantially same latencies to a particular image store, the imageprocessing system 125 can select one of the two image feeding serversthat has a lesser work load, or in other words, has a higher capacity.

In some embodiments, the server score is determined as a function of thecomputing resources of an image feeding server, e.g., processor speed,available memory. In some embodiments, the lesser the computingresources of an image feeding server, the lesser the server score of theimage feeding server.

Various combinations of the above factors can be considered indetermining the server score of the image feeding server. Further, eachof the above factors can be weighted, that is, different factors canhave different weights in determining the score. After the scores of allthe image feeding servers are determined, the image processing system125 selects an image feeding server whose server score is the highest orexceeds a predefined threshold and instructs the selected image feedingserver to obtain the source image from the image store, downsize thesource image and transmit the downsized source image to the imageprocessing system.

The image processing system 125 can implement various methods forselecting an image feeding server using the score. In a first method,the image processing system 125 generates a job request for obtaining asource image from a particular image store, e.g., image store 120. Theimage processing system 125 broadcasts a request for response requestingthe image feeding servers to respond with their scores for the jobrequest. Each of the image feeding servers 150 determine their imagefeeding server parameter values for the job request, e.g., latency withthe image store 120, workload and computing resources, calculate theirscores as a function of the image feeding server parameters, andtransmit their scores to the image processing system 125. In someembodiments, the image feeding servers may transmit the image feedingserver parameter values instead of the scores to the image processingsystem 125, and the image processing system 125 may determine the scoresusing the image feeding server parameter values. The image processingsystem 125 identifies an image feeding server, e.g., image feedingserver 150 a, that has the highest server score or a server score whichexceeds the predefined threshold, and instructs the identified imagefeeding server to transmit the downsized source image to the imageprocessing system 125.

In a second method, the image feeding servers 150 may advertise theirscores or the image feeding server parameter values to the imageprocessing system 125 on a periodic basis or based on a trigger. Thetrigger can be occurrence of an event, such as expiry of an intervalsince the last transmission of the score, receiving a request for theserver score from the image processing system 125, the work load of theimage feeding server dropping below a predefined threshold, etc. In someembodiments, since the latency of an image feeding server can vary fordifferent image stores, the server score of the image feeding server canalso vary. Accordingly, an image feeding server can determine its serverscore (or image feeding server parameter values) for each of the imagestores in the system 200, generate a set of scores (or image feedingserver parameter values) and transmit the set of scores to the imageprocessing system 125. Each of the image feeding servers can transmittheir corresponding set of scores to the image processing system 125.When the image processing system 125 generates a job request forobtaining a source image from a particular image store, the imageprocessing system 125 checks the scores stored at the image processingsystem 125 and identifies an image feeding server that has the highestserver score or a server score which exceeds the predefined thresholdfor the particular image store. The image processing system 125 theninstructs the identified image feeding server obtain the source imagefrom the particular image store, downsize the source image and transmitthe downsized source image to the image processing system 125.

In some embodiments, the job request can also include a target file sizeof the downsized source image, that is, the file size to which thesource image has to be downsized. The image processing system 125determines the target file size such that the downsized source image ofthe target file size can be used to generate a number of processedimages that may be served to the end user. In some embodiments, thetarget file size is determined such that the downsized image is largeenough to be used for generating a combination of processed images thatmay be requested by the client system for serving to the end user andsmall enough to be transmitted from the image store to the imageprocessing system 125 without consuming significant computing resources.

The image processing system 125 communicates with the image feedingservers 150 over a first communication network 230 and the image feedingservers 150 communicate with image stores over communication networks235 a-c (referred to as second communication network 235). Both thecommunication networks have low latencies. In some embodiments, thefirst communication network 230 is a high speed network and therefore,the latency between with the image processing system 125 and the imagefeeding servers 150 is relatively low. Also, in some embodiments, sincean image feeding server that is in proximity to a particular image storeis chosen to obtain the source image from the particular image store,the latency between the image feeding server and the particular imagestore can be significantly less compared to the latency of the networkbetween the image processing system 125 and the image stores in theabsence of the image feeding servers 150.

Further, the combined latencies of both the communication networks isstill less compared to the latency between the image processing system125 and the image stores in the absence of the image feeding servers150. Accordingly, the computing resources consumed in having one of theimage feeding servers 150 obtain the source image from the image store,downsize the source image and transmit the downsized source image to theimage processing system 125 can be significantly less compared to thatconsumed in having the image processing system 125 obtain the sourceimage in original file size from the image store without the imagefeeding servers 150.

FIG. 3 is a block diagram of an image processing system of the FIG. 1.The image processing system 125 includes a request dispatch module 305,a server determination module 310, an image receiving module 315, animage processing module 320, a server score determination module 325 anda request receiving module 330. In some embodiments, the imageprocessing system 125 is implemented as a distributed computing system.

The request receiving module 330 receives a request from a clientsystem, e.g., client system 115, to serve a processed image to an enduser device accessing the content containing the processed image. Theimage processing module 320 determines whether a source image requiredfor generating the processed image is stored at the image processingsystem 125. If the source image is stored at the image processing system125, the image processing module 320 processes the source image togenerate the processed image and serves the processed image to the enduser device. If the source image is not stored at the image processingsystem 125, the request dispatch module 305 generates a job request forobtaining a downsized source image.

The job request can include location information of the image store of aclient system from which an image feeding server has to obtain thesource image. In some embodiments, the location information can includeIP (Internet Protocol) address of an image store such as image store120. The job request can also include a target file size to which thesource image obtained from the image has to be downsized to before beingtransmitted to the image processing system 125. The request dispatchmodule 305 dispatches the job request to multiple image feeding servers,e.g., image feeding servers 150, seeking their response to the jobrequest.

The server determination module 310 determines an image feeding serverto which the job request has to be assigned based on the responsesreceived from the image feeding servers. The server determination module310 selects an image feeding server based on a server score of the imagefeeding server. The server determination module 310 selects an imagefeeding server having a server score that is highest among the scores ofall the image feeding servers or that exceeds a predefined threshold.

The response from the image feeding servers can include scores of theimage feeding servers or image feeding server parameters that is used todetermine the scores. In some embodiments, the responses include scoresof the image feeding servers. The server determination module 310retrieves the scores from the responses and identifies the image feedingserver with the highest server score or a server score that exceeds apredefined threshold. In some embodiments, the responses include imagefeeding server parameters, e.g., a latency of the image feeding serverto the particular image feeding server, a workload of the image feedingserver or a computing resource of the image feeding server. The serverdetermination module 310 retrieves the image feeding server parametervalues and requests a server score determination module 325 to determinea server score of each of the servers as a function of the image feedingserver parameter values. After determining the scores, the serverdetermination module 310 identifies the image feeding server with thehighest server score or a server score that exceeds a predefinedthreshold.

Referring back to the server score determination module 325, the serverscore determination module 325 sends a message to each of the imagefeeding servers 150 to determine a server score and report back theserver score to the image processing system 125. In some embodiments,the server score determination module 325 requests the image feedingservers to determine a server score of the corresponding image feedingserver based on latency between the corresponding server and the imagestorage of the client system. In some embodiments, the server scoredetermination module 325 requests the image feeding servers to determinea server score of the corresponding server based on a number ofdownsizing jobs being performed by the corresponding server and a filesize of a specified source image which is being downsized in each of thedownsizing jobs.

After the server determination module 310 identifies the image feedingserver, the request dispatch module 305 assigns the request to theidentified image feeding server. The image feeding server obtains thesource image from the particular image store, downsizes the source imageand transmits the source image to the image processing system 125. Theimage receiving module 315 receives the downsized source image from theimage feeding server and stores the downsized source image at a storagedevice associated with the image processing system 125.

After receiving the downsized source image, the image processing module320 processes the downsized source image to generate a processed imagebased on the client system, which requested the processed image, and/ora context of the request from the end user device accessing the contentcontaining the processed image.

FIG. 4 is a flow diagram of a process for processing an image,consistent with various embodiments. In some embodiments, the process400 may be implemented in an environment such as environment 100 ofFIG. 1. At step 405, a request receiving module 330 of an imageprocessing system 125 receives a request from a client system, such asclient system 115, for obtaining a processed image for a particularimage. In some embodiments, the request can be originated from an enduser device, such as end user device 105 a accessing content publishedby the client system 115. For example, the request can be generated whenthe end user device 105 a accesses a URL of a website of the clientsystem 115 having the processed image.

At determination step 410, the image processing module 320 determines ifa source image required for generating the processed image is stored atthe image processing system 125. Responsive to a determination that thesource image is not stored at the image processing system 125, at step415, the request dispatch module 305 sends a request to an image feedingserver to obtain the source image from the image store where the clientsystem stores the original source images, and the process 400 proceedsto step 420. In some embodiments, the image feeding server downsizes thesource image obtained from the image store before transmitting it to theimage processing system 125. Responsive to a determination that thesource image is stored at the image processing system 125, at step 420,the image processing module 320 identifies a recipe for generating theprocessed image and processes the downsized source image based on therecipe to generate the processed image.

In some embodiments, processing the image includes rendering the imagebased on (a) a target size, (b) a target resolution, (c) a target colorpattern, (d) a target layout of the image with respect to other contentwith which the image is served, (e) a target brightness value, (f) atarget contrast value, or (g) a target saturation value. The processingcan also include highlighting a target portion of the image or croppinga target area of the image, recognizing a face in the image, performingpredefined image editing operations on the recognized face etc.

Further, the above processing can also be dependent on a context of therequest from the end user, e.g., an attribute of the end user device 105a such as (a) a type of the end user device 105 a, (b) a size of adisplay of the end user device 105 a, (c) a resolution of the display ofthe end user device 105 a, (d) an operating system of the end userdevice 105 a, (e) ambient lighting of the end user device 105 a etc. Forexample, if the end user device 105 a has a display with a resolutionhigher than a predefined resolution, then the recipe can includeinstructions to render the image in high resolution. In another example,if the end user device 105 a has a display with resolution lower than apredefined resolution, then the recipe can include instructions torender the image in low resolution. In another example, the brightnessof the image can be changed based on the ambient lighting condition onthe end user device 105 a.

At step 425, the image processing module 320 transmits the processedimage to the client system 115, which serves the processed image to theend user device 105 a. For example, the processed image is served withthe content in response to the end user device 105 a accessing the URLof the website of the client system 115.

FIG. 5 is a flow diagram of a process for selecting an image feedingserver to obtain the downsized source image, consistent with variousembodiments. In some embodiments, the process 500 may be implemented inthe environment 100 of FIG. 1. At step 505, the request dispatch module305 of an image processing system 125 generates a job request forobtaining the downsized source image. In some embodiments, the requestdispatch module 305 generates the job request at step 415 of process400, when the image processing module 320 determines that a source imagerequired for generating the processed image is not stored at the imageprocessing system 125.

The job request can include location information of the image store of aclient system, such as image store 120 associated with client system115, from which an image feeding server has to obtain the source image.In some embodiments, the location information can include IP address ofthe image store. The job request can also include a target file size towhich the source image obtained from the image has to be downsized tobefore being transmitted to the image processing system 125. The requestdispatch module 305 dispatches the job request to multiple image feedingservers, e.g., image feeding servers 150, seeking their response to thejob request.

At step 510, the server determination module 310 receives responses fromthe image feeding servers. The response from the image feeding serverscan include scores of the image feeding servers or image feeding serverparameters that is used to determine the scores. The server score isindicative of a match of the image feeding server to perform the jobadvertised. The server score can be determined as a function of one ormore image feeding server parameters, e.g., latency, workload, computingresource, of an image feeding server, which is further described atleast with reference to process 700 of FIG. 7 below. In someembodiments, the responses include scores of the image feeding servers.In some embodiments, the responses include image feeding serverparameters, e.g., a latency of the image feeding server to theparticular image feeding server, a workload of the image feeding serveror a computing resource of the image feeding server. The serverdetermination module 310 retrieves the image feeding server parametervalues and requests a server score determination module 325 to determinea server score of each of the servers as a function of the image feedingserver parameter values.

After the scores are retrieved from the responses or determined based onthe image server parameter values in the responses, at step 515, theserver determination module 310 identifies the image feeding server withthe highest server score or a server score that exceeds a predefinedthreshold.

At step 520, the request dispatch module 305 assigns the request to theidentified image feeding server. The image feeding server obtains thesource image from the particular image store, downsizes the source imageand transmits the source image to the image processing system 125. Atstep 525, the image receiving module 315 receives the downsized sourceimage from the image feeding server and stores the downsized sourceimage at a storage device associated with the image processing system125.

FIG. 6 is a flow diagram of a process 600 for feeding a downsized sourceimage to an image processing system, consistent with various embodimentsof the disclosed technology. The process 600 can be executed in anenvironment 100 of FIG. 1. At step 605, each of the image feedingservers, e.g., image feeding servers 150, receive a job requestdispatched from the request dispatch module 305. As described withreference to step 605 of process 600, the job request can includelocation information of the source image, location information of theimage store of a client system, e.g., IP address of the image store,from which an image feeding server has to obtain the source image. Thejob request can also include a target file size to which the sourceimage obtained from the image store has to be downsized to before beingtransmitted to the image processing system 125.

At step 610, each of the image feeding servers determine a server scoreor image feeding server parameters used to generate the server score forservicing the job request. The server score can be determined as afunction of one or more image feeding server parameters, e.g., latency,workload, computing resource, of an image feeding server, which isfurther described at least with reference to process 700 of FIG. 7below. At step 615, the image feeding servers send their scores or imagefeeding server parameters to the image processing system 125.

The image processing system 125 determines the image feeding server withthe highest server score or a server score that exceeds a predefinedthreshold and the image processing system 125 assigns the job request tothe identified image feeding server. At step 620, the image feedingserver obtains a confirmation from the image processing system forservicing the job request. At step 625, the image feeding server obtainsthe source image from the image store. At step 630, the image feedingserver downsizes the source image to the target file size and at step635, transmits the downsized source image to the image processing system125.

FIG. 7 is a flow diagram of a process 700 for determining a server scoreof an image feeding server, consistent with various embodiments of thedisclosed technology. The process 700 can be executed in an environment100 of FIG. 1. A server score of the image feeding server is indicativeof a match of the image feeding server to perform the job advertised bythe image processing system.

The server score can be determined as a function of one or more imagefeeding server parameters. At step 705, the one or more image feedingserver parameters are determined. In some embodiments, the image feedingserver parameters can include latency, workload, or computing resourceof an image feeding server. Determining the server parameters caninclude determining a latency between a particular image feeding serverand a particular image store from which the source image is to beobtained (step 706), determining a workload (or available capacity) ofthe image feeding server (step 707) and determining computing resourcesof the image feeding server, e.g., processor speed, available memory(step 708). It should be noted that the image feeding server parametersdiscussed in steps 706-708 are examples, and that other image feedingserver parameters or job based parameters, e.g., file size of the sourceimage to be fetched, can be considered for determining the server scoreof the image feeding server.

After the image feeding server parameters are determined, at step 710,the server score of the image feeding server is determined as a functionof the image feeding server parameters. Each of the image feeding serverparameters can be weighted, that is, different image feeding serverparameters can have different weights in determining the score.

Referring back to determining the latency of an image feeding server,latency is a time interval between the stimulation and response, or atime taken for a data packet to travel from source to destination andback. In some embodiments, the lower the latency the higher the score.For example, for obtaining a source image from image store 120, theimage feeding server 150 a can be the most proximate image feedingserver to the image store 120 and therefore, can have a least latency tothe image store 120 among the image feeding servers 150 andconsequently, the highest score. In some embodiments, the server scoreof an image feeding server can be different for different image storesas the image feeding server can have different latencies for differentimage stores.

Referring back to determining the workload of an image feeding server,the work load of the image feeding server can be determined as afunction of number of tasks, e.g., downsizing tasks, the image feedingserver is performing and a file size of the source image being downsizedin each of the downsizing tasks. For example, if two image feedingservers have substantially same latencies to a particular image store,the image processing system 125 can select one of the two image feedingservers that has a lesser work load, or in other words, has a highercapacity to perform the current job request. The lesser the work load ofthe image feeding server, the higher the server score of the imagefeeding server.

Referring back to determining the computing resources of an imagefeeding server, the lesser the available computing resources at an imagefeeding server, the lesser the server score of the image feeding server.For example, in determining a server score for two image feedingservers, one with a faster processor and/or more memory than the otheris likely to have a higher server score than the other.

Note that the determination of server score can be performed by eithereach of the image feeding servers or the image processing system 125. Ifthe image feeding servers are determining their scores, steps 705-710may be performed by each of the image feeding servers. If the imageprocessing system is determining the score, each of the image feedingservers may determine their corresponding image feeding serverparameters, e.g., as described in steps 705-708, and transmit them tothe image processing system, which determines the scores based on theimage feeding server parameters.

FIG. 8 illustrates a block diagram of an image processing system 800implemented as a multi-tier architecture, consistent with variousembodiments. The image processing system 800 includes various servertiers such as edge server tier 805, a shield server tier 820, loadbalancing server tier 835, an image processing server tier 850 and asource image server tier 865. In some embodiments, the image processingsystem 125 of FIG. 1 can be implemented as the image processing system800, and in the environment 100.

Each of the tiers can have one or more servers. A server can be either aphysical computer in a data center, or a virtual instance of a computerin cloud infrastructure. A server typically receives requests for work,performs or distributes the work, and then returns a response. The edgeserver tier 805 can receive requests from a client system such as clientsystem 115 (e.g., on behalf of an end user) for obtaining processedimages. For example, when the end user accesses the content published bythe client system 115, which includes the processed image, the edgeserver tier 805 receives the request from the client system 115 toobtain the processed image. The edge server tier 805 includes multipleservers, such as edge server 810, and one or more associated caches,such as edge cache 815. In some embodiments, each of the edge servers inthe edge server tier 805 has an associated edge cache as illustrated inFIG. 8. However, in some embodiments, the edge cache 815 may beassociated with more than one edge server.

In some embodiments, the edge servers in the edge server tier 805 may belocated in various geographical locations. The edge servers may beconfigured to serve requests from predefined geographical locations.When a request arrives at the edge server tier 805 from a client system,the request is routed to an edge server that is configured to serve thegeographical location in which the particular client system is located.For example, while edge server 810 may be configured to serve requestsfrom geographical locations A, B and C, another edge server may beconfigured to serve requests from geographical locations X, Y, andanother edge server may be configured to serve requests fromgeographical location Z. That is, the edge servers in the edge servertier 805 act as a closest point of contact of the image processingsystem 800 for the client systems.

When a request is received at the edge server tier 805 from the clientsystem 115 at a geographical location A, the edge server 810 checkswhether the edge cache 815 has the processed image. If the edge cache815 has the processed image, the edge server 810 serves the processedimage to the client system 115, which further serves the processed imageto the end user's device, such as end user device 105 a. If the edgecache 815 does not contain the processed image, the edge server tier 805passes the request to the shield server tier 820.

The shield server tier 820 includes multiple servers, such as shieldserver 825, and associated shield cache 830. In some embodiments, theshield servers in the shield server tier 820 can be distributed acrossvarious geographical regions. In some embodiments, a geographical regionincludes a number of geographical locations. The shield servers can beconfigured to receive requests from a group of edge servers in aparticular geographical region. In some embodiments, the shield servertier 820 can have a shield server for every “x” number of edge servers,where “x” is a number configurable by a user, such as an administratorof the image processing system 800. The shield cache 830 can beassociated with one or more shield servers, and the shield server tier820 can have one or more shield caches.

When a shield server, such as shield server 825, receives the requestfrom an edge server, such as edge server 810, the shield server 825determines whether the shield cache 830 contains the processed image. Ifthe shield cache can contains the processed image, the shield server 825obtains the processed image from the shield cache 830 and returns theimage to the edge server 810 which further serves the processed image tothe client system 115. If the shield cache 830 does not contain theprocessed image, the shield server 825 passes the request to the loadbalancing server tier 835.

In some embodiments, the edge server tier 805 can bypass the shieldserver tier 820 and pass the request to the load balancing server tier835 directly if the processed image is not available at the edge cache815. However, typically, the edge server tier 805 determines whether theprocessed image is available at the shield server tier 820 before therequest is passed to the load balancing server tier 835. In someembodiments, the time consumed for the determination, by the edgeserver, of whether a shield server contains the processed image is notsignificant because the edge server and the shield server can share thesame communication network.

The load balancing server tier 835 receives the image processingrequests 845 from the edge server tier 805 and/or shield server tier 820and inserts them into a queue 840. An image processing server of theimage processing server tier 850 fetches a request from the queue 840and processes the request to generate a processed image. In someembodiments, the load balancing server tier 835 balances a load of theimage processing system 800 by inserting the requests into the queue 840and not assigning them directly to the image processing servers in theimage processing server tier 850 upon receipt of a request at the loadbalancing server tier 835. The image processing server can pick up arequest from the queue 840 when the image processing server is availableto process a request.

In some embodiments, the load balancing server tier 835 determineswhether a particular request is a valid request for an image beforeinserting the request into the queue 840. In some embodiments, thevalidity of the request can be determined by determining whether asignature of an image for which the request is received matches with theany of the signatures maintained at the image processing system 800. Ifthe particular request is valid, the load balancing server tier 835inserts it into the queue 840.

The image processing server tier 850 obtains an image processing request(also referred to as “a task”) for a particular image from the queue 840and processes the particular image based on a recipe for the particularimage and the context of the request. The image processing server tier850 determines whether the particular image, that is, an original sourceimage provided by the client system 115 is available at the imageprocessing server cache 860. If the original source image is availableat the image processing server cache 860, the image processing servertier 850 obtains the image from the image processing server cache 860and processes the original source image to generate the processed image.If the original source image is not available at the image processingserver cache 860, the image processing server cache 860 requests thesource image server tier 865 to obtain the original source image. Thesource image server tier 865 obtains the original source image andreturns it to the image processing server tier 850 for processing theoriginal source image. The image processing server tier 850 may storethe received original source image at the image processing server cache860.

In some embodiments, the source image server tier 865 obtains theoriginal source image from a location specified by the client system115, for example, at storage such as the image store 120. The sourceimage server tier 865 obtains the original source image from the imagestore 120 via image feeding servers, e.g., image feeding servers 150,and returns the original source image to the image processing servertier 850. In some embodiments, the source image server tier 865 itselfcan be the image feeding servers 150. In some embodiments, the sourceimage server tier 865 obtains a downsized version of the source image.The source image server tier 865 can also store the retrieved originalsource image at the source image cache 870. In some embodiments,obtaining the original source image from the image store 120 can consumesignificant amount of computing resources, especially in cases where theimage store 120 is implemented as cloud storage service, for a number ofreasons including performing authentication operations, performing readoperations from a slow storage medium such as a hard drive etc.Accordingly, in order to minimize the consumption of computingresources, the original source image retrieved by the source imageserver tier 865 may be stored at the source image cache 870.

Also, in some embodiments, the source image server tier 865 may,regardless of whether a request is received for obtaining a particularoriginal source image, pre-fetch the original images from the imagestore 120 and store the original source images in the source image cache870. The pre-fetching may be performed based on a predefined condition,for example, at a particular time of the day, day of the week, etc., orwhen the load on the image processing system 800 is below a predefinedthreshold. In some embodiments, the pre-fetching may help the sourceimage server tier 865 to minimize the delay caused due to obtaining aparticular source image from the image store 120, in serving a requestfor the particular original source image from the image processingserver tier 850. The source image server tier 865 may refresh the sourceimage cache 870, that is, add or delete original source images from thesource image cache 870 based on predefined cache management policies.

Referring back to the image processing server tier 850, after obtainingthe original source image, either from the image processing server cache860 or from the source image server tier 865, the image processingserver tier 850 identifies a recipe for the original source image andprocesses the original source image using the recipe and a context ofthe image processing request to generate a processed image. The recipecan be identified using various data including information regarding theclient, the content being published (e.g., based on URL of a webpage ofthe content), information regarding attributes of an end user devicefrom which the request is initiated etc. Any necessary information fordetermining the recipe for processing the original source image isprovided to the image processing system 800, for example, via therequest. In some embodiments, the information for determining the recipemay be provided by the client system 115 to the image processing system800 independent of the request.

The recipe can be provided by the client and/or generated by the imageprocessing system 800. The recipes can be stored at a storage systemsuch as recipe storage system 855. In some embodiments, the recipestorage system 855 can be implemented as various storage systems, forexample, a database, a file system and/or any other convenient or knownstorage techniques.

The recipe includes a set of instructions to be executed on the originalsource image to render a processed image. The recipe can includeinstructions regarding at least one of (a) a target size of the image tobe rendered, (b) a target resolution of the image to be rendered, (c) atarget color pattern of the image to be rendered, (d) a target layout ofthe image with respect to other content with which the image is served,(e) a target brightness value of the image to be rendered, (f) a targetcontrast value of the image to be rendered, (g) a target saturationvalue of the image to be rendered, (h) a target portion of the image tobe highlighted, or (i) a target area of the image to be cropped etc.

In some embodiments, the recipe can also be based on a context of therequest. The context of the request can include an attribute of the enduser device 105 a. The attribute of the end user device 105 a caninclude at least one of (a) a type of the end user device 105 a, (b) asize of a display of the end user device 105 a, (c) a resolution of thedisplay of the end user device 105 a, (d) an operating system of the enduser device 105 a, or (e) ambient lighting of the end user device 105 aetc. In some embodiments, the recipe can be determined based on thecontext of the request. For example, if the end user device 105 a has adisplay with a resolution higher than a predefined resolution, then therecipe can include instructions to render the image in high resolution.In another example, if the end user device 105 a has a display withresolution lower than a predefined resolution, then the recipe caninclude instructions to render the image in low resolution. In anotherexample, the brightness of the image can be changed based on the ambientlighting condition on the end user device 105 a.

Further, in some embodiments, the context of the request can alsoinclude network bandwidth availability between the end user device 105 aand the client system 115. For example, if the network bandwidth isbelow a predefined threshold, the recipe can include instructions torender a low resolution image.

After the original image is processed based on the recipe and thecontext of the request, the processed image is returned to the clientsystem 115 which further serves the processed image to the end userdevice 105 a.

In some embodiments, when the image processing server tier 850 returnsthe processed image to the client system 115, the processed image istransmitted to the client system 115 though one or more tiers, e.g.,load balancing server tier 835, shield server tier 820 and edge servertier 805, above the image processing server tier 850. The one or moretiers may also store the received processed image in a cache associatedwith the corresponding server tier.

In some embodiments, the image processing server tier 850 can includedifferent servers for performing different parts of processing theimage. For example, tasks such as face recognition can be performed by aset of servers, changing a color scheme of a specific portion can beperformed by another set of servers, cropping can be performed byanother set of servers etc. The image processing server tier 850 canhave logic, either in software, hardware, firmware or a combination, toassign a particular image processing operation to a particular server inthe image processing server tier 850.

Further, in some embodiments, a particular image processing request canbe split into a number of sub-tasks. For example, if a particularrequest requires face recognition and cropping, the face recognitionoperation and the cropping operation can be split into two tasks andadded to the queue 840 as two sub-tasks. The face recognition task canbe assigned to a server that performs face recognition task and thecropping task can be assigned to a server that performs the croppingoperation. The final processed image can be rendered by combining theresults of the sub-tasks or having each successive task work on theimage resulting from the previous sub-task until all the sub-tasks arecompleted.

In some embodiments, one or more tiers of the image processing system800 generate log 880 containing information at least regarding imageprocessing requests and the image processing operations performed on theimages. The log 880 can include information such as whether a particularprocessed image was available in a cache associated with a particulartier or was retrieved from a cache associated with a tier below theparticular tier; performance metrics on rendering the processed image,e.g., time taken to render the processed image, memory consumed, a sizeof the original source image, how many source images were used togenerate the processed image, time taken to load, wait or convert aparticular image, a total response time in serving the processed imageto the client system 115, response time of each tier, average time aparticular task was in the queue 840, a number of tasks waiting in thequeue at a given point of time, a point in time when the maximum queuesize was reached, a time when evicting queue items were started etc.

In some embodiments, each of the one or more tiers can generate the log880 separately. In some embodiments, the one or more tiers can generateinformation into a single log file 880. The log 880 includes informationto identify which of the tiers generated particular information in thelog 880. In some embodiments, the image processing system 800 may use alogging system (not illustrated), e.g., real-time logging systems, togenerate the log 880. The logging system can be provided by a thirdparty.

In some embodiments, the information in the log 880 can be analyzed andused to enhance the performance of the image processing system 800. Insome embodiments, the information in the log 880 can be used to predictimage processing requests that may be received in the future andpre-render the processed images in advance. The pre-rendered images maybe stored at a cache, such as edge server cache 815. When a predictedrequest is actually received at the edge server tier 805, the edgeserver 810 may obtain the processed image from the edge server cache 815and return the processed image to the client system 115. By predictingthe requests, processing the predicted request to rendered the processedimage and storing the processed image at the cache in advance ofreceiving the actual request, the time consumed in obtaining theprocessed image when the actual request is received is minimizedsignificantly. This enhances the user experience of the end user sincethe end user experiences minimum to near zero delay in the processedimage being rendered on the end user device 105 a.

In some embodiments, the image processing system 800 includes predictionservers 885 that generate predictions 890 having information regardingfuture image processing requests. The predictions 890 are input to theimage processing system 800 via the edge server tier 805 (similar to theimage processing requests from client system 115) which renders theprocessed images based on the predictions 890. In some embodiments, theimage processing system 800 may schedule rendering of the processedimages for the predictions 890 based on a predefined criterion. Forexample, image processing system 800 may decide to render the processedimages for the predictions 890 when a load on the image processingsystem 800 is below a predefined threshold, at a particular time of theday, a particular time of the week, when a predefined number of requestshave been received from a particular client system, etc.

The prediction servers 885 can generate the predictions 890 based on theinformation in the log 880 using various known prediction techniques.For example, the predictions 890 can be generated based on a statisticalanalysis of the information in the log 880. In another example, theprediction servers 885 can employ an expert system or a rule engine thatcan help in generating the predictions 890 based on the information inthe log 880.

In some embodiments, one or more tiers of the image processing system800 can be owned or operated by different entities. That is, the imageprocessing system 800 can be a combination of various infrastructureprovided by different entities.

FIG. 9 is a block diagram of a computer system as may be used toimplement features of some embodiments of the disclosed technology. Thecomputing system 900 may be used to implement any of the entities,components or services depicted in the examples of FIGS. 1-8 (and anyother components described in this specification). The computing system900 may include one or more central processing units (“processors”) 905,memory 910, input/output devices 925 (e.g., keyboard and pointingdevices, display devices), storage devices 920 (e.g., disk drives), andnetwork adapters 930 (e.g., network interfaces) that are connected to aninterconnect 915. The interconnect 915 is illustrated as an abstractionthat represents any one or more separate physical buses, point to pointconnections, or both connected by appropriate bridges, adapters, orcontrollers. The interconnect 915, therefore, may include, for example,a system bus, a Peripheral Component Interconnect (PCI) bus orPCI-Express bus, a HyperTransport or industry standard architecture(ISA) bus, a small computer system interface (SCSI) bus, a universalserial bus (USB), IIC (I2C) bus, or an Institute of Electrical andElectronics Engineers (IEEE) standard 1394 bus, also called “Firewire”.

The memory 910 and storage devices 920 are computer-readable storagemedia that may store instructions that implement at least portions ofthe described technology. In addition, the data structures and messagestructures may be stored or transmitted via a data transmission medium,such as a signal on a communications link. Various communications linksmay be used, such as the Internet, a local area network, a wide areanetwork, or a point-to-point dial-up connection. Thus, computer-readablemedia can include computer-readable storage media (e.g.,“non-transitory” media) and computer-readable transmission media.

The instructions stored in memory 910 can be implemented as softwareand/or firmware to program the processor(s) 905 to carry out actionsdescribed above. In some embodiments, such software or firmware may beinitially provided to the processing system 900 by downloading it from aremote system through the computing system 900 (e.g., via networkadapter 930).

The technology introduced herein can be implemented by, for example,programmable circuitry (e.g., one or more microprocessors) programmedwith software and/or firmware, or entirely in special-purpose hardwired(non-programmable) circuitry, or in a combination of such forms.Special-purpose hardwired circuitry may be in the form of, for example,one or more ASICs, PLDs, FPGAs, etc.

Remarks

The above description and drawings are illustrative and are not to beconstrued as limiting. Numerous specific details are described toprovide a thorough understanding of the disclosure. However, in certaininstances, well-known details are not described in order to avoidobscuring the description. Further, various modifications may be madewithout deviating from the scope of the embodiments. Accordingly, theembodiments are not limited except as by the appended claims.

Reference in this specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the disclosure. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment, nor are separate or alternative embodimentsmutually exclusive of other embodiments. Moreover, various features aredescribed which may be exhibited by some embodiments and not by others.Similarly, various requirements are described which may be requirementsfor some embodiments but not for other embodiments.

The terms used in this specification generally have their ordinarymeanings in the art, within the context of the disclosure, and in thespecific context where each term is used. Certain terms that are used todescribe the disclosure are discussed below, or elsewhere in thespecification, to provide additional guidance to the practitionerregarding the description of the disclosure. For convenience, certainterms may be highlighted, for example using italics and/or quotationmarks. The use of highlighting has no influence on the scope and meaningof a term; the scope and meaning of a term is the same, in the samecontext, whether or not it is highlighted. It will be appreciated thatthe same thing can be said in more than one way. One will recognize that“memory” is one form of a “storage” and that the terms may on occasionbe used interchangeably.

Consequently, alternative language and synonyms may be used for any oneor more of the terms discussed herein, nor is any special significanceto be placed upon whether or not a term is elaborated or discussedherein. Synonyms for certain terms are provided. A recital of one ormore synonyms does not exclude the use of other synonyms. The use ofexamples anywhere in this specification including examples of any termdiscussed herein is illustrative only, and is not intended to furtherlimit the scope and meaning of the disclosure or of any exemplifiedterm. Likewise, the disclosure is not limited to various embodimentsgiven in this specification.

Those skilled in the art will appreciate that the logic illustrated ineach of the flow diagrams discussed above, may be altered in variousways. For example, the order of the logic may be rearranged, substepsmay be performed in parallel, illustrated logic may be omitted; otherlogic may be included, etc.

Without intent to further limit the scope of the disclosure, examples ofinstruments, apparatus, methods and their related results according tothe embodiments of the present disclosure are given below. Note thattitles or subtitles may be used in the examples for convenience of areader, which in no way should limit the scope of the disclosure. Unlessotherwise defined, all technical and scientific terms used herein havethe same meaning as commonly understood by one of ordinary skill in theart to which this disclosure pertains. In the case of conflict, thepresent document, including definitions will control.

1-20. (canceled)
 21. A method comprising: receiving, at an imageprocessing system, a first request from a client system for serving afirst image to a user device; receiving, at the image processing system,an intermediate image from a first server, wherein the receivingincludes: transmitting, by the image processing system, a second requestfor response to each of multiple servers; receiving, by the imageprocessing system, a response from each server of a subset of themultiple servers for performing a task on a source image, the responseincluding a corresponding score indicative of a match of correspondingserver of the multiple servers to perform the task on the source image;sending, by the image processing system, a message to the first serverto perform the task; processing, by the image processing system, theintermediate image to generate the first image; and transmitting, by theimage processing system, the first image to the client system forfurther transmission to the user device.
 22. The method of claim 21,wherein the corresponding score of each corresponding server isdetermined as a function of latency between the corresponding server andan image store from which the corresponding server is to obtain thesource image.
 23. The method of claim 21, wherein the first serverincludes a first score that satisfies a selection criterion.
 24. Themethod of claim 22, wherein sending the message to perform the taskincludes sending the message to: obtain the source image from the imagestore associated with the client system, the source image having a firstfile size, and downsize the source image to generate the intermediateimage, the intermediate image having a second file size, the second filesize being smaller than the first file size.
 25. The method of claim 21,wherein the intermediate image has a resolution less than that of thesource image.
 26. The method of claim 21, wherein the intermediate imageis a file format different from that of the source image.
 27. The methodof claim 21, wherein the intermediate image has lesser number of colorsthan in the source image.
 28. The method of claim 21, wherein theintermediate image is a compressed file whereas the source image is nota compressed file.
 29. The method of claim 21, wherein receiving thefirst request from the client system includes: receiving the firstrequest at the image processing system in response to a user requestreceived at the client system from the user device.
 30. The method ofclaim 21, wherein the first server has a first latency with the imagestore and a second latency with the image processing system, the firstlatency being lower than the second latency.
 31. The method of claim 21,wherein the image processing system has a first latency with the firstserver and a second latency with the image store, the first latencybeing lower than the second latency.
 32. The method of claim 21, whereinprocessing the intermediate image to generate the first image includes:processing, by the image processing system and based on a recipeprovided by the client system, the intermediate image to generate thefirst image.
 33. A non-transitory computer-readable storage mediumstoring computer-readable instructions, comprising: instructions forreceiving, at an image processing system, a first request from a clientsystem for serving a first image to a user device; instructions forreceiving, at the image processing system, an intermediate image from afirst server, wherein the receiving includes: instructions fortransmitting, by the image processing system, a second request forresponse to each of multiple servers; instructions for receiving, by theimage processing system, a response from each server of a subset of themultiple servers for performing a task on a source image, the responseincluding a corresponding score indicative of a match of correspondingserver of the multiple servers to perform the task on the source image;instructions for sending, by the image processing system, a message tothe first server to perform the task; instructions for processing, bythe image processing system, the intermediate image to generate thefirst image; and instructions for transmitting, by the image processingsystem, the first image to the client system for further transmission tothe user device.
 34. The non-transitory computer-readable storage mediumof claim 33, wherein the corresponding score of each correspondingserver is determined as a function of latency between the correspondingserver and an image store from which the corresponding server is toobtain the source image.
 35. The non-transitory computer-readablestorage medium of claim 33, wherein the corresponding score isindicative of an amount of computing resources consumed by thecorresponding server in performing the task.
 36. The non-transitorycomputer-readable storage medium of claim 33, wherein the correspondingscore of each corresponding server is determined as an average of a loadof the corresponding server and the latency of the server.
 37. Thenon-transitory computer-readable storage medium of claim 33, wherein theload of the corresponding server is determined as a function of a numberof tasks being performed by the corresponding server and a file size ofthe source image on which the task is to be performed by thecorresponding server.
 38. The non-transitory computer-readable storagemedium of claim 33, wherein the first server includes a first score thatsatisfies a selection criterion, and wherein the first score isindicative of a first amount of computing resources consumed by thefirst server in performing the task, the first amount of computingresources being an amount of computing resources to be consumed by anyof a remaining of the servers in performing the task.
 39. An apparatuscomprising: a processor; a request receiving module configured toreceive a first request from a client system for serving a first imageto a user device; a request dispatch module configured to transmit asecond request for response to each of multiple servers; a serverdetermination module configured to receive a response from each serverof a subset of the multiple servers for performing the task, theresponse including a corresponding score indicative of a match of acorresponding server of the multiple server to perform the task on thesource image, wherein the request dispatch module is further configuredto send a message to a first server to perform the task; and an imageprocessing module configured to: receive an intermediate image from thefirst server, process the intermediate image to generate the firstimage, and transmit the first image to the client system for furthertransmission to the user device.
 40. The apparatus of claim 39, whereinthe corresponding score of each corresponding server is determined as afunction of latency between the corresponding server and an image storefrom which the corresponding server is to obtain the source image. 41.The apparatus of claim 39, wherein the first server includes a firstscore that satisfies a selection criterion.
 42. The apparatus of claim39, wherein the image processing system has a first latency with thefirst server and a second latency with the image store, the firstlatency being lower than the second latency.
 43. The apparatus of claim39, wherein processing the intermediate image to generate the firstimage includes: process the intermediate image to generate the firstimage, the context including attributes associated with the user devicefor which the first image is to be transmitted.
 44. The apparatus ofclaim 39, wherein the image processing module is further configured to:process the intermediate image to generate multiple processed imagesbased on multiple image processing instructions, the processed imagesbeing part of content associated with the client system, each of theprocessed images associated with a corresponding image processinginstruction.
 45. A method comprising: receiving, at an image processingsystem, a first request from a client system for serving a first imageto a user device; sending, by the image processing system, a message tothe first server to perform a task on a source image; receiving, at theimage processing system, an intermediate image from a first server, theintermediate image comprising the performed task on the source image;processing, by the image processing system, the intermediate image togenerate the first image; and transmitting, by the image processingsystem, the first image to the client system for further transmission tothe user device.
 46. The method of claim 45, wherein the first server isselected based on a corresponding score of each corresponding serverthat is determined as a function of latency between the correspondingserver and an image store from which the corresponding server is toobtain the source image.
 47. The method of claim 45, wherein the firstserver includes a first score that satisfies a selection criterion. 48.A non-transitory computer-readable storage medium storingcomputer-readable instructions, comprising: instructions for receiving,at an image processing system, a first request from a client system forserving a first image to a user device; instructions for selecting, atthe image processing system, a first server of multiple servers toperform a task and generate an intermediate image based on acorresponding score indicative of a match of each server of the multipleservers to perform the task on the source image; instructions forsending, by the image processing system, a message to the first serverto perform the task; instructions for receiving, at the image processingsystem, the intermediate image from the first server; instructions forprocessing, by the image processing system, the intermediate image togenerate the first image; and instructions for transmitting, by theimage processing system, the first image to the client system forfurther transmission to the user device.